Blog search

Friday Facts #344 - Tile transition collisions & Team Steelaxe speedrun record

Posted by Klonan, Bilka on 2020-04-24

Tile transition collisions Klonan We first mentioned a change to our tile transition logic back in FFF-199, and not long after in FFF-214. These two posts focused more on the visual side, and how it makes the game terrain look so much better. In short, the tile transition logic overlays an additional sprite over adjacent tiles, so that where the two tiles meet has a much more natural look. Left: Tile transitions on; Right: Tile transitions off. So while the looks were taken care of, we also had to deal with the 'feel' of the tiles. The easiest example of this is the 1x1 landfill 'stepping stones'. It really looks like you should be able to walk/drive across the 1 tile of water. So we added in an additional layer of collision checks, which will consider the transitions when performing the logic of what can go where. Now some of the cheesier among you will know that biters don't know how to get across these 1 tile gaps. That is because simply we never enabled the biters to use this collision check logic. One reason is that more checks means more UPS usage for the biter pathfinding, another is that we didn't think it was necessary. However it was available in the engine, and any mod could enable it if they want. That is exactly what I did with my Mining drones mod. Initially this seemed to work, and I thought it might make them walk around lakes a bit more naturally (like the player character does). However quickly I noticed that people were reporting on the forum that the game was crashing with the mod installed. I quickly reverted the change to my mod and we started looking into it. It turns out that the new abstract pathfinder we added for better unit pathfinding (FFF-317) was not set up to consider units using this tile transition collision logic. This same crash was happening sometimes without any mods installed, but the case was more difficult to reproduce, so this is a nice situation where mods help us work on issues in the base game. Recently I have been working on another unit heavy mod, Transport drones, and the principle design behind the mod heavily relies on the tile collision logic (the units don't even collide with entities). It turned out to be a really nice test of our new pathfinder, but also highlighted some of the issues that not considering the tile transitions can bring. By not considering the tile transitions, the drone takes an awkward path along the diagonal road. This week Oxyd finished his work on an upgrade to the pathfinder, so we can enable the tile transition collision logic with units. The change is immediately noticeable with the above example. By considering the tile transitions, the drones path much more naturally. With the logic now in place, we are debating whether to enable it for biters or not. We probably won't, it is only a minor change and would have a non-zero performance impact (a rough test puts the worst case at about 5%), but then again it is a fun way to surprise those who thought the 1 tile of water would stop the biters attacking, and it kinda makes sense they can walk over it just as the player can. Well we will see if there are any issues with it in the modded cases before any further consideration. As a bonus fact (this is Factorio facts after all), I spent a bit of time benchmarking some late game Transport drone based factories (screenshot), and found a few nice performance gains.

Friday Facts #288 - New remnants, More bugs

Posted by Wheybags, Rseding, Dom, Albert on 2019-03-29

Removing RTL language translations Wheybags I'm sorry to say that we have removed the RTL language translations (Hebrew and Arabic) in 0.17.20. Until this point we've had a half implementation of RTL languages, where the text is simply flipped when we download it from Crowdin. This 'works' for a decent proportion of things, but not nearly 100%. In order to attain the level of polish we want for the 1.0 release, we would need to spend a lot of time implementing proper support for RTL layouts. This just doesn't make sense for us given our current goals, and the proportion of our player base which uses these languages (less than 0.1%). We decided that instead of completely gutting the translations, we could leave them in for those who enjoy them, but not to offer them in the GUI as defaults. The languages will remain up on Crowdin, and the locale files will still be present in game, but there will be no option in the in-game language options dialog to choose them. If you want to use an RTL language, you will have to manually edit your config file to set your locale. Detailed instructions are available on our forum. What this also means, is that we won't be investigating any bug reports about RTL issues.

Friday Facts #312 - Fluid mixing saga & Landfill terrain

Posted by Dominik, Ernestas, Albert on 2019-09-13

Fluid mixing saga Dominik Hello Factorians, Today I would like to talk to you about my favourite subject - fluid mixing and its prevention. It is a new mechanic introduced in 0.17 that seemed quite simple at first, but has been giving me nightmares ever since. A while ago I took on the task of updating the fluid system (FFF-260). The way it works in 0.16 is that the fluid boxes (the thing that holds the fluid and is contained in entities like pipes or refineries) had no organisation whatsoever except their connections. They would sit there and do nothing and then once per update send fluid somewhere. It had it's issues especially with symmetry, and when you happened to put some fluid where it did not belong, you could end up requiring major demolition works. My task was to develop a better algorithm to move the fluids, and a very very optional side task I had in mind was to do something about the fluids getting to wrong places and mixing. We started by organizing the fluid boxes into systems (connected FBs make a system) managed by a special fluid manager, and optimizing the heck out of it (FFF-271). Soon after, I started working on the new algorithm and anti fluid-mixing. Until then nobody really considered preventing the fluid mixing seriously as it did not seem possible. But when I thought about it, it did not seem that hard. The idea is to simply not allow any action that would lead to fluids getting where they are not supposed to go. When you build a steam pipeline, you should not need nor want it to touch another fluid. In principle, it would only require a few quite realistic mechanisms: A connected block of fluid boxes would either be fluid-free, or it would be locked to one fluid. Two such blocks can connect only if they are compatible - either one has no fluid lock, or they have the same fluid. An assembler can only set a recipe if it is compatible with the fluid locks on its fluid boxes. Have a migration from old saves that can contain mixing. By creating the fluid systems right before, number 1 was almost finished and we only needed to assign the lock. It would be set either by a fluid, or by a 'filter', which is a fluidbox that is set to use a fluid - such as a water pump using water, or assembler having some inputs/outputs given by a recipe. After a while I had both the fluid algorithm and the mixing done (FFF-274). The mixing was not that easy (like 5x more complicated) but it worked pretty well. As for the fluid algorithm, V453000 and Twinsen found some issues with waves on a macro scale, and because it was right before releasing the 0.17 experimental, we decided to hold it off on it for the time being (we have a new version now that seems okay, but has to wait for 0.17 becoming stable first). The mixing made it through though and seemed quite finished. It turned out that the work on it was at most one tenth complete. Some difficulties appeared already before the initial release, but that was only a hint of what would come later. One by one, then ten by ten, bugs started coming. The problem is that as often as not, they were not just little issues with simple fixes. Instead, over and over, they turned the current solution on its head and the code had to be completely rewritten in order to account for the new case. As of now, almost exactly half a year and many rewrites later, it is still not fully bug free. The number one issue was with Assembling machines. It turned out that their code was pretty messy and does not simply allow the checks I needed, so it had to be refactored. The next problem was that the check was not only required when setting a recipe on an existing assembler - as I naively thought - but in about one million different situations I would not dream existed. Building a fixed recipe assembler. Reviving an assembler. Rotating. Blueprint of an assembler. Blueprint with a recipe and a rotation placed over a ghost with a non-default rotation during a full moon. Teleport. Script building. Copy-paste settings. Any combination of these. Pretty much every case would go through different paths in the code and behave entirely differently. Fixing one would break another and so tons of tests had to be written. When these cases finally worked, it turned out that doing these operations when they fail (e.g. can’t rotate because it would cause mixing) brings another wave of issues. And then mods come and the complexity multiplies. All this, although in a more simple form, had to be done for all other entities that can use fluid too, such as inserters (mods...). Another number one issue are underground connections. When playing, you would not think twice about them but on a closer look they are all but simple. They have this (cough) uncomfortable feature (want to shoot myself) that you can build another underground pipe between them (or take it out) and a complex reconnection logic jumps in. It means that based on building order, connections are established and reestablished differently. This is a big thing when building a blueprint with many pipes, or loading a save game. But the largest issue is one tiny corner case with huge implications. Have two underground pipes that have different fluids, but are split by a third, and it gets removed. Until now mixing was theoretically completely avoidable, but here it was and there was nothing to stop it. As a result, a new concept is introduced - a blocked connection. An underground connection that would normally connect but can’t. Establishing it is the easier part. But when does it get fixed? An entity miles away gets rotated, that splits a fluid system, disconnecting the blocked connection from a fluid filter on the other side, and the connection should unblock. But even something as simple as a rotation contains fluid fixes and re-establishing of fluid boxes and if it fails, it still fixes the blocked connection in the process and it can’t be undone... You get the picture. And we are still talking about underground pipes, while anything can have underground connections, including said assemblers, which the previous code did not consider at all. The complexity just goes deeper and deeper, and Rseding is probably right that we should have never gone this way at all. So the bug fixing has been basically running in circles - clearing one batch of bugs, thinking that those were the last ones and the ordeal is finally over, only to find another batch (ideally from some bizzare modder idea) a few days later. Many times I wished that mods never existed. Nor multiplayer, or any players at all for that matter. About two months ago all seemed really fine, with basically no reports and just a few crashes in the automated crash reports. At that moment another nightmare materialized in the form of a talented volunteer bug tester named boskid, who took on a personal crusade to break the fluids to dust (I actually imagine him sitting in his dark room with evil laughter, dreaming about making my next day worse than the previous). In all seriousness, he has done a great deal of work with his bug testing, creating bizzare modded cases and testing scripts, and deserves a big thanks. He is actually coming to our offices next week, so follow the news for reports of developers falling out of windows. An example of a nice configuration he found (and I can’t fix). The whole time we were taking the offensive approach to mixing - crash the game when it happens - so that we know about bugs to fix them. Recently we have decided to put a stop to it and have the mixing automatically fix itself once it is detected, eyeing the end of this episode. Right now I have 3 mixing bugs on the list and I am sure those are the last, and mixing will be done (lying to myself is a way to cope with it) and the new fluid algorithm can come soon after :).

Friday Facts #156 - Massive Multiplayer

Posted by Tomas on 2016-09-16

Hi all, sometimes, writing FFF feels like spitting out blood - there is simply very little to write about. Yet the Factorio blog subscriber crowd is there and waiting ... Sometimes it is the opposite and there is abundance of interesting topics available. Today it is the latter case. Enjoy!

Friday Facts #200 - Plans for 0.16

Posted by kovarex & TOGoS on 2017-07-21

Hello, I can't believe that we have been able to produce a post every Friday for 200 weeks without missing a single one. To be honest I'm not sure if this isn't the right time to pause for a while, to avoid being this kind of show that gets worse and worse over time until it is so bad that you want to take your intestines and strangle yourself with them. But people in the office convinced me with arguments like "FFF is the only good thing we have", so we probably have to continue for a little longer.

Friday Facts #326 - Particle emitter & Data cache

Posted by Allaizn, Rseding, Klonan on 2019-12-20

More particle optimisations Allaizn Rseding's recent optimisations of the particle system (FFF-322) made particles much more lightweight than they were before, but it still left particles as rather complex beasts. A quick summary of the possible actions a particle can make during it's update: Move their own position. Advance their animation to another frame. Land in water and apply a trigger. Apply another trigger with a certain frequency. Remove themselves from the game world once their life time ends. What makes this complex is that triggers are general purpose systems that can do all kinds of things, including creating and destroying entities, fire, smoke and other particles as well as playing sounds or recursively applying even more triggers. In other words: applying a trigger is an "anything can happen" situation and thus totally unpredictable, which in turn makes optimisations extremely hard.

Friday Facts #302 - The multiplayer megapacket

Posted by Twinsen, Klonan on 2019-07-05

The multiplayer megapacket Twinsen Last month I joined KatherineOfSky's MMO event as a player. I noticed that after we reached a certain number of players, every few minutes a bunch of them got dropped. Luckily for you (but unluckily for me), I was one of the players who got disconnected every, single. time, even though I had a decent connection. So I took the matter personally and started looking into the problem. After 3 weeks of debugging, testing and fixing, the issue is finally fixed, but the journey there was not that easy. Multiplayer issues are very hard to track down. Usually they only happen under very specific network conditions, in very specific game conditions (in this case having more than 200 players). Even when you can reproduce the issue it's impossible to properly debug, since placing a breakpoint stops the game, messes up the timers and usually times out the connection. But through some perseverance and thanks to an awesome tool called clumsy, I managed to figure out what was happening. The short version is: Because of a bug and an incomplete implementation of the latency state simulation, a client would sometimes end up in a situation where it would send a network package of about 400 entity selection input actions in one tick (what we called 'the megapacket'). The server then not only has to correctly receive those input actions but also send them to everyone else. That quickly becomes a problem when you have 200 clients. It quickly saturates the server upload, causes packet loss and causes a cascade of re-requested packets. Delayed input actions then cause more clients to send megapackets, cascading even further. The lucky clients manage to recover, the others end up being dropped. The issue was quite fundamental and took 2 weeks to fix. It's quite technical so I'll explain in juicy technical details below. But what you need to know is that since Version 0.17.54 released yesterday, multiplayer will be more stable and latency hiding will be much less glitchy (less rubber banding and teleporting) when experiencing temporary connection problems. I also changed how latency hiding is handled in combat, hopefully making it look a bit smoother.

Friday Facts #274 - New fluid system 2

Posted by Dominik, Klonan, kovarex on 2018-12-21

New Fluid system 2 (Dominik) Hi Factorians, Here is Dominik, with an update on the fluids. This time it is pretty much finished so I can tell you facts instead of just speculations. You will find how the new algorithm will work and some new handy usability features. In FFF-260 I wrote about how it all started, why we are doing it and what the plan is. There was a huge response from you all and I want to thank everyone for their contributions. Let me apologise to redditors, as at the beginning I started responding on the forums and when I realized there is reddit too, there were too many comments for me to handle. The forum users produced many ideas on how the system could work. About third of them was a fluid teleportation, many where known but many were entirely new and interesting. What intrigued me was the large variety of backgrounds they came from - differents kinds of engineers (mechanical, CS, electrical, ...), mathematicians, physicists, and even people with real pipes hands on experience. I won’t go through them here, you can find them on the forums or reddit. There were two proposals on the forum though that were so good that they made it into the game - from quinor and TheYeast. Both of these proposals were very similar and kinda similar to the previous game logic. What it shares is that the mechanic still uses fluid physics simulation and volume in a pipe as a base for the movement calculation. As a result, not much changes on the first glance. What they add though is an emphasis on the fluid network update being independent on the current state (i.e. updating one pipe only depends on state from the last tick) and is therefore independent on evaluation order, which was one of the big pains of the old model that led to sometimes ridiculous junction behavior. Difference between these two was rather small - quinor’s version allowed perfect throughput with 3 passes over the fluidboxes (fluidbox is the thing managing fluids for entities, so I will talk about them), while Yeast’s one was 2 pass with ¼ throughput. What was outstanding though is that TheYeast, a physicist, supported the model with a nice theoretical background and what’s more, he made an amazing JS simulator to test and compare various modification of the model. Because that extra pass in quinor’s version was too high a price for the perfect throughput, I went with TheYeast’s two pass one. Since the old algorithm only used a single pass run by entities for the update, I first needed to overhaul the whole system to allow accommodating the new one. Going from one pass to two passes necessarily means higher complexity, so we made a big effort to optimize everything we could to make sure we will still end up faster than 0.16. Kovarex wrote about it in FFF-271.

Friday Facts #264 - Texture streaming

Posted by posila on 2018-10-12

Hello, it is me, posila, with another technical article. Sorry.

Friday Facts #263 - Trains in blueprints

Posted by kovarex on 2018-10-05

Trains in blueprints Building trains again and again might be a daunting task. Especially when you start making a lot of mining outposts, artillery/supply trains with filtered cargo wagon slots etc. So I decided that we should extend blueprints to work with trains as well. The first condition was, that trains are only selected when you explicitly allow it in the checkbox, so they don't get in your way when building rail setups. Checking the button allows the train that was there to be put into the blueprint (similar to the way tiles work). For the sake of simplicity, we decided that once there is any rail in the blueprint, the train in it will be always buildable (as a ghost obviously), even if there are not rails to support the train at the moment. The train ghost will simply stay there and won't be buildable until rails are placed under it in a way so it can be placed. If I remove the rails from the blueprint, I get a second type of rail blueprint. In this case, all the parts of need to have rails to support it, this is mainly needed as without rails, there is no rail grid forced, so we should make sure, the train ghost won't be created in some wrong position. The small touch here is, that the blueprint also contains the schedule. With little-bit of improvisation, I can optimize the mine building a lot in the late game. I create a blueprint of mine train station. The stop will be called " Mine X". Both of the trains in the blueprint will have the " Mine X" -> " Smelting" schedule setup. Once I build the blueprint, I just rename the " Mine X" to whatever I want (" Mine 12" for example), and the train schedules are updated as well, so I'm almost ready to go. The last tweak I'm considering is to allow blueprints to contain the fuel insertion info similar to how they contain the module insertion info for assembling machines now.